Search for: All records

Multi-agent reinforcement learning (MARL) lies at the heart of a plethora of applications involving the interaction of a group of agents in a shared unknown environment. A prominent framework for studying MARL is Markov games, with the goal of finding various notions of equilibria in a sample-efficient manner, such as the Nash equilibrium (NE) and the coarse correlated equilibrium (CCE). However, existing sample-efficient approaches either require tailored uncertainty estimation under function approximation, or careful coordination of the players. In this paper, we propose a novel model-based algorithm, called VMG, that incentivizes exploration via biasing the empirical estimate of the model parameters towards those with a higher collective best-response values of all the players when fixing the other players’ policies, thus encouraging the policy to deviate from its current equilibrium for more exploration. VMG is oblivious to different forms of function approximation, and permits simultaneous and uncoupled policy updates of all players. Theoretically, we also establish that VMG achieves a near-optimal regret for finding both the NEs of two-player zero-sum Markov games and CCEs of multi-player general-sum Markov games under linear function approximation in an online environment, which nearly match their counterparts with sophisticated uncertainty quantification. 

Abstract Dust from core-collapse supernovae (CCSNe), specifically Type IIP supernovae (SNe IIP), has been suggested to be a significant source of the dust observed in high-redshift galaxies. CCSNe eject large amounts of newly formed heavy elements, which can condense into dust grains in the cooling ejecta. However, infrared (IR) observations of typical CCSNe generally measure dust masses that are too small to account for the dust production needed at high redshifts. Type IIn SNe (SNe IIn), classified by their dense circumstellar medium, are also known to exhibit strong IR emission from warm dust, but the dust origin and heating mechanism have generally remained unconstrained because of limited observational capabilities in the mid-IR (MIR). Here, we present a JWST/MIRI Medium Resolution Spectrograph spectrum of the SN IIn SN 2005ip nearly 17 yr post-explosion. The SN IIn SN 2005ip is one of the longest-lasting and most well-studied SNe observed to date. Combined with a Spitzer MIR spectrum of SN 2005ip obtained in 2008, this data set provides a rare 15 yr baseline, allowing for a unique investigation of the evolution of dust. The JWST spectrum shows the emergence of an optically thin silicate dust component (≳0.08M_⊙) that is either not present or more compact/optically thick in the earlier Spitzer spectrum. Our analysis shows that this dust is likely newly formed in the cold, dense shell (CDS), between the forward and reverse shocks, and was not preexisting at the time of the explosion. There is also a smaller mass of carbonaceous dust (≳0.005M_⊙) in the ejecta. These observations provide new insights into the role of SN dust production, particularly within the CDS, and its potential contribution to the rapid dust enrichment of the early Universe. 

Abstract We present new JWST/MIRI Medium Resolution Spectroscopy and Keck spectra of SN 1995N obtained in 2022–2023, more than 10,000 days after the supernova (SN) explosion. These spectra are among the latest direct detections of a core-collapse SN, both through emission lines in the optical and thermal continuum from infrared (IR) dust emission. The new IR data show that dust heating from radiation produced by the ejecta interacting with circumstellar matter is still present but greatly reduced from when SN 1995N was observed by the Spitzer Space Telescope and WISE in 2009/2010 and 2018, when the dust mass was estimated to be 0.4M_⊙. New radiative-transfer modeling suggests that the dust mass and grain size may have increased between 2010 and 2023. The new data can alternatively be well fit with a dust mass of 0.4M_⊙and a much reduced heating source luminosity. The new late-time spectra show unusually strong oxygen forbidden lines, stronger than the Hαemission. This indicates that SN 1995N may have exploded as a stripped-envelope SN, which then interacted with a massive H-rich circumstellar shell, changing it from intrinsically Type Ib/c to Type IIn. The late-time spectrum results when the reverse shock begins to excite the inner H-poor, O-rich ejecta. This change in the spectrum is rarely seen but marks the start of the transition from SN to SN remnant.